Image sensor and image capturing apparatus

ABSTRACT

An image sensor includes a pixel portion in which a plurality of unit pixels each having one micro lens and a plurality of photoelectric conversion portions are arrayed in a matrix, a signal readout portion that reads out signals accumulated in the photoelectric conversion portions and converts the read signals to digital signals, a signal processor that processes signals read out by the signal readout portion and has an image capture signal processor that performs signal processing for generating a captured image on signals read out by the signal readout portion and a focus detection signal processor that performs signal processing for focus detection on signals read out by the signal readout portion, and an output portion that outputs signals processed by the signal processor.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of application Ser. No. 17/329,608,filed May 25, 2021, which is a continuation of application Ser. No.16/596,086, filed Oct. 8, 2019, which issued as U.S. Pat. No. 11,048,063on Jun. 29, 2021, which is a continuation of application Ser. No.15/750,547, filed Feb. 6, 2018, which issued as U.S. Pat. No. 10,514,526on Dec. 24, 2019, which is a national stage of International ApplicationNo. PCT/JP2016/003900, filed Aug. 26, 2016 the entire disclosures ofwhich is hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to an image sensor and an image capturingapparatus.

BACKGROUND ART

In recent years, advancements in the high-functionality andmulti-functionality of image capturing apparatuses that use a CMOS imagesensor or the like have been made to support various needs. Increases inpixel count and advancements in high-speed imaging have been made toCMOS image sensors, and there is increasing demand for methods thatallow for faster reading out of pixel signals.

For example, a method for performing high-speed reading out that hasbeen in widespread use in recent years involves disposing ananalog-to-digital conversion circuit (hereinafter, column ADC) everycolumn and performing digital output, as disclosed in Japanese PatentLaid-Open No. 2005-278135. By introducing column ADCs, it becomespossible to perform digital transmission of pixel signals to outside ofthe image sensor, and high-speed readout has become possible followingtechnical improvements in digital signal transmission.

On the other hand, as an example of multi-functionalization, an imagecapturing apparatus capable of, for example, acquiring not only theintensity distribution of light but also the incident direction anddistance information of light has been proposed. Japanese Patent No.3774597 discloses an image sensor capable of focus detection usingsignals obtained from the image sensor. The image sensor has aconfiguration in which a photodiode (hereinafter, PD) corresponding toone micro lens is divided in two, such that each PD receives light froma different pupil surface of the taking lens. Focus detection isperformed by comparing the outputs of the two PDs. A normal capturedimage can also be obtained by adding together the output signals fromthe two PDs constituting a unit pixel.

An image capturing apparatus disclosed in Japanese Patent Laid-Open No.2009-89105 is provided with a mode for reading out focus detectionsignals, exposure control signals and image capture signals for liveview display from a solid-state image sensor with one vertical scan ofthe same frame. The image capturing apparatus disclosed in JapanesePatent Laid-Open No. 2009-89105 is described as being able to performlive view display with faster focus detection control and exposurecontrol.

However, since the signals of all of the PDs need to read out in thecase of performing focus detection and exposure control in an imagesensor such as disclosed in Japanese Patent No. 3774597 and JapanesePatent Laid-Open No. 2009-89105, there is a problem in that the timerequired to read out the signals of the PDs increases and the frame ratedecreases. Even if the signal readout time is reduced by a readoutmethod using column ADCs such as in Patent Document 1, further increasesin pixel count and frame rate are expected in the future, and thusfurther shortening of the signal readout time is desired.

SUMMARY OF INVENTION

The present invention has been made in view of the abovementionedproblems, and provides an image sensor that is able to greatly shortenthe time taken in order for signals required in drive control of animage capturing apparatus, such as focus detection signals, to be outputfrom the image sensor.

According to a first aspect of the present invention, there is providedan image sensor comprising: a pixel portion in which a plurality of unitpixels each having one micro lens and a plurality of photoelectricconversion portions are arrayed in a matrix; a signal holding portionconfigured to hold signals output from the unit pixels of an entire areaof the pixel portion; a signal processor configured to process signalsheld by the signal holding portion, and having an image capture signalprocessor configured to perform signal processing for generating acaptured image on signals held by the signal holding portion and a focusdetection signal processor configured to perform signal processing forfocus detection on signals held by the signal holding portion; and anoutput portion configured to output signals processed by the signalprocessor.

According to a second aspect of the present invention, there is providedan image capturing apparatus comprising: an image sensor; an imageprocessor configured to process signals output from the image sensor andgenerate an image; and a focus detector configured to process readsignals and perform focus detection, wherein the image sensor includes:a pixel portion in which a plurality of unit pixels each having onemicro lens and a plurality of photoelectric conversion portions arearrayed in a matrix; a signal holding portion configured to hold signalsoutput from the unit pixels of an entire area of the pixel portion; asignal processor configured to process signals held by the signalholding portion, and having an image capture signal processor configuredto perform signal processing for generating a captured image on signalsheld by the signal holding portion and a focus detection signalprocessor configured to perform signal processing for focus detection onsignals held by the signal holding portion; and an output portionconfigured to output signals processed by the signal processor.

According to a third aspect of the present invention, there is providedan image sensor comprising: a pixel portion in which a plurality ofpixels configured to photoelectrically convert light from a subject aredisposed; a readout portion configured to read out signals from thepixel portion; and an output portion configured to output, among thesignals read out by the readout portion, the signals of the pixels of anentire area of the pixel portion to outside of the image sensor, assignals for generating an image, and the signals of the pixels of apartial area of the pixel portion to outside of the image sensor, assignals for calculating an evaluation value to be used in drive controlof an apparatus that includes the image sensor.

According to a fourth aspect of the present invention, there is providedan image sensor comprising: a pixel portion in which a plurality ofpixels configured to photoelectrically convert light from a subject aredisposed; a readout portion configured to read out signals from thepixel portion; a phase difference detection portion configured tocalculate a phase difference evaluation value to be used in focusdetection control employing a phase difference detection method; and anoutput portion configured to output, among the signals read out by thereadout portion, the signals of the pixels of an entire area of thepixel portion to outside of the image sensor, as signals for generatingan image, and the phase difference evaluation value calculated by thephase difference detection portion using the signals of the pixels of apartial area of the pixel portion to outside of the image sensor.

According to a fifth aspect of the present invention, there is providedan image capturing apparatus comprising: an image sensor including: apixel portion in which a plurality of pixels configured tophotoelectrically convert light from a subject are disposed; a readoutportion configured to read out signals from the pixel portion; and anoutput portion configured to output, among the signals read out by thereadout portion, the signals of the pixels of an entire area of thepixel portion to outside of the image sensor, as signals for generatingan image, and the signals of the pixels of a partial area of the pixelportion to outside of the image sensor, as signals for calculating anevaluation value to be used in drive control of an apparatus thatincludes the image sensor; and a selector configured to select thepartial area.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the configuration of an imagecapturing apparatus according to a first embodiment of the presentinvention.

FIG. 2 is a diagram illustrating the configuration of unit pixels of animage sensor of the first embodiment.

FIG. 3 is a conceptual view in which light beams emitted through an exitpupil of a taking lens are incident on a unit pixel.

FIG. 4 is a block diagram showing the configuration of the image sensorof the first embodiment.

FIG. 5 is a diagram illustrating a pixel circuit and a readout circuitof an image sensor of the first embodiment.

FIG. 6 is a timing chart showing a signal readout operation of the imagesensor of the first embodiment.

FIG. 7 is a diagram illustrating the configuration of a signal processorin the image sensor of the first embodiment.

FIG. 8A is a diagram showing focus detection signal output areas in apixel area of a second embodiment.

FIG. 8B is a diagram showing focus detection signal output areas in apixel area of a second embodiment.

FIG. 9 is a block diagram showing the configuration of an imagecapturing apparatus according to a third embodiment of the presentinvention.

FIG. 10 is a flowchart showing the flow of processing for image captureplane phase difference AF in the third embodiment.

FIG. 11 is a diagram showing a screen for selecting a focus detectionarea in the third embodiment.

FIG. 12A is a diagram showing a subject detection result screen in thethird embodiment.

FIG. 12B is a diagram showing a subject detection result screen in thethird embodiment.

FIG. 13 is a diagram illustrating the configuration of an image sensorof a fourth embodiment.

FIG. 14 is a diagram illustrating a pixel circuit and readout circuitsof the image sensor of the fourth embodiment.

FIG. 15 is an overall configuration diagram of an image sensor of afifth embodiment.

FIG. 16 is an overall configuration diagram of the image sensor of thefifth embodiment.

FIG. 17 is an overall configuration diagram of an image sensor of asixth embodiment.

FIG. 18 is a block diagram showing the configuration of an imagecapturing apparatus according to a seventh embodiment.

FIG. 19 is a block diagram showing the configuration of an imagecapturing apparatus according to an eighth embodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing the configuration of an imagecapturing apparatus having an image sensor of the first embodiment ofthe present invention. An image sensor 100 is configured to be providedwith a light receiving portion 102, a readout portion 103, a controller104, a signal processor 105, and an output portion 106. The lightreceiving portion 102 has a plurality of unit pixels that are disposedin a matrix, and receives the light of an optical image formed by thetaking lens 101. The configuration of the light receiving portion 102will be discussed later. The readout portion (A/D conversion portion)103, on receipt of a drive control signal of the controller 104, A/Dconverts image signals that are output from the light receiving portion102, and sends the A/D converted image signals to the signal processor105.

The signal processor 105 performs computational operation processingsuch as addition, subtraction and multiplication of signals, processingfor selecting signals to be output to the outside from the image sensor100 via the output portion 106, and the like on the A/D converted imagesignals. Also, the signal processor 105 performs processing includingvarious types of correction such as reference level adjustment and thelike, and rearrangement of data. This processing is performed on receiptof a control signal from the controller 104. The signal processor 105,which will be discussed in detail later, performs image capture signalprocessing and focus detection signal processing on image signalsobtained from the light receiving portion 102, and sends the processedimage signals to the output portion 106. The output portion 106 outputsthe image signals processed by the signal processor 105 to outside ofthe image sensor 100.

The image processor 107 receives image capture signals from the outputportion 106 of the image sensor 100, performs image processing such asdefective pixel correction, noise reduction, color conversion, whitebalance correction and image correction, resolution conversionprocessing and image compression processing, and generates still imagesor moving images. A phase difference detection portion 108, on receiptof a focus detection signal from the output portion 106, calculates aphase difference evaluation value for performing focus detection.

An overall control/operation unit 109 performs overall drive and controlof the image sensor 100 and the entire image capturing apparatus. Adisplay unit 110 displays shot images, live view images, various settingscreens, and the like. A recording unit 111 and a memory unit 112 arerecording media such as a nonvolatile memory or a memory card thatrecord and hold image signals output from the overall control/operationunit 109, and the like. An operation unit 113 receives commands from auser using an operation member provided to the image capturingapparatus, and inputs the commands to the overall control/operation unit109. A lens controller 114 calculates optical system drive informationbased on the phase difference evaluation value calculated by the phasedifference detection portion 108, and controls the position of a focuslens of the taking lens 101.

Next, the relationship between the taking lens 101 in the imagecapturing apparatus of the present embodiment and the light receivingportion 102 of the image sensor 100, the definition of a pixel, and theprinciples of focus detection using a pupil division method will bedescribed.

FIG. 2 is a schematic diagram showing the configuration of unit pixels200 of the image sensor 100. In FIG. 2 , a micro lens 202 furtherfocuses light formed into an image on the image sensor 100 by the takinglens 101 on a pixel-by-pixel basis. Photoelectric conversion portions201A and 201B composed of photodiodes (PDs) receive light incident onthe unit pixel 200, and generate and accumulate signal charge thatdepends on the amount of received light. As a result of the unit pixel200 having two photoelectric conversion portions under one micro lens202, the two photoelectric conversion portions 201A and 201B are eachcapable of receiving light that has passed through an exit pupil areadivided in two. The signal obtained by combining the two signals of thephotoelectric conversion portions 201A and 201B on a pixel-by-pixelbasis is the output signal of one pixel serving as an image generationpixel. Also, focus detection of the taking lens 101 can be performed bycomparing the signals that are obtained from the two photoelectricconversion portions on a pixel-by-pixel basis. That is, focus detectionemploying a phase difference detection method according to which thepupil is divided in the left-right direction is possible by performing acorrelation operation on the signal that is obtained from thephotoelectric conversion portion 201A and the signal that is obtainedfrom the photoelectric conversion portion 201B in a certain area withinthe unit pixel 200.

FIG. 3 is a diagram in which light that has passed through the takinglens 101 passes through the single micro lens 202 and is receiving bythe unit pixel 200 of the light receiving portion 102 of the imagesensor 100 as observed from a direction perpendicular (Y-axis direction)to the optical axis (Z-axis). Light that has passed through exit pupils302 and 303 of the taking lens is incident on the unit pixel 200 so asto be centered on the optical axis. At this time, the amount of incidentlight is adjusted by a lens diaphragm 301. As shown in FIG. 3 , thelight beam that passes through the pupil area 302 passes through themicro lens 202 and is received by the photoelectric conversion portion201A, and the light beam that passes through the pupil area 303 passesthrough the micro lens 202 is received by the photoelectric conversionportion 201B. Accordingly, the photoelectric conversion portions 201Aand 201B each receive light from different areas of the exit pupil ofthe taking lens.

The signal of the photoelectric conversion portion 201A that pupildivides the light from the taking lens 101 is acquired from a pluralityof unit pixels 200 arranged side-by-side in the X-axis direction, andthe subject image constituted by this group of output signals is givenas an A image. The signal of the photoelectric conversion portion 201Bthat similarly pupil divides the light from the taking lens 101 isacquired from a plurality of unit pixels 200 arranged side-by-side inthe X-axis direction, and the subject image constituted by this group ofoutput signals is given as a B image.

A correlation operation is implemented on the A image and the B image,and the amount of image shift (pupil division phase difference) isdetected. Furthermore, by multiplying the amount of image shift by aconversion factor that is determined from the optical system and thefocal position of the taking lens 101, a focal position corresponding toan arbitrary subject position within the screen can be calculated. Bycontrolling the focal position of the taking lens 101 based on focalposition information calculated here, image capture plane phasedifference AF (autofocus) becomes possible. Also, by giving a signalobtained by adding together the A image signal and the B image signal asan A+B image signal, this A+B image signal can be used as a normal shotimage.

Next, the configurations of the light receiving portion 102 and thereadout portion 103 of the image sensor 100 will be described usingFIGS. 4 and 5 . FIG. 4 is a block diagram showing an exemplaryconfiguration of the light receiving portion 102 and the readout portion103 of the image sensor 100. The light receiving portion 102 has a pixelportion 401 and a drive circuit portion 402. A plurality of unit pixels200 are arrayed horizontally (row direction) and vertically (columndirection) in the pixel portion 401. Although a total of six unit pixels200 (2 rows×3 columns) are illustrated in FIG. 4 , millions or tens ofmillions of unit pixels 200 are actually disposed. The drive circuitportion 402 includes a power supply circuit, a timing generator (TG), ascanning circuit, and the like for driving the pixel portion 401. Bydriving the pixel portion 401 using the drive circuit portion 402, pixelsignals of the entire image capture area of the pixel portion 401 areoutput from the pixel portion 401 to the readout portion 103. The drivecircuit portion 402 is driven on receipt of a control signal from thecontroller 104 of FIG. 1 . Analog-to-digital conversion (A/D conversion)is performed on the pixel signals from the pixel portion 401 that areinput to the readout portion 103. Note that the readout portion 103 isconfigured to be provided with a plurality of readout circuits, such asone readout circuit per column, for example.

Incidentally, with regard to the method of driving the pixel portion401, noise such as crosstalk and blooming tends to occur when rows thatare adjacent to each other are driven under different conditions (framerate, accumulation time, etc.). However, in the present embodiment,since the drive circuit portion 402 drives the entire area of the pixelportion 401 uniformly under the same conditions, such problems do notarise.

FIG. 5 is a diagram showing an example of the unit pixel 200 of theimage sensor 100 and a readout circuit 509 constituting the readoutportion 103. In the unit pixel 200, a transfer switch 502A is connectedto the photoelectric conversion portion 201A composed of a photodiode(PD), and a transfer switch 502B is connected to the photoelectricconversion portion 201B. Charge generated by the photoelectricconversion portions 201A and 201B is respectively transferred to acommon floating diffusion portion (FD) 504 via the transfer switches502A and 502B, and is temporarily saved. The charge transferred to theFD 504 is output to a column output line 507 as a voltage correspondingto the charge via an amplification MOS transistor (SF) 505 forming asource follower amplifier when a selection switch 506 is turned on. Acurrent source 508 is connected to the column output line 507.

A reset switch 503 resets the potential of the FD 504 to VDD, and thepotentials of the photoelectric conversion portions 201A and 201B to VDDvia the transfer switches 502A and 502B. The transfer switches 502A and502B, the reset switch 503, and the selection switch 506 arerespectively controlled by control signals PTXA, PTXB, PRES and PSEL,via signal lines that are connected to the drive circuit portion 402,which on the periphery thereof.

Next, the circuit configuration of the readout circuit 509 will bedescribed. An amplifier 510 amplifies the signal output by the columnoutput line 507, and a capacitor 512 is used in order to hold the signalvoltage. Writing to the capacitor 512 is controlled by a switch 511 thatis turned on and off using a control signal PSH. A reference voltageVslope supplied from a slope voltage generation circuit (not shown) isinput to one input of a comparator 513, and the output of the amplifier510 written to the capacitor 512 is input to the other input. Thecomparator 513 compares the output of the amplifier 510 and thereference voltage Vslope, and outputs one of two values, namely, lowlevel and high level, depending on the magnitude relationshiptherebetween. Specifically, low level is output in the case where thereference voltage Vslope is smaller than the output of the amplifier510, and high level is output in the case where the reference voltageVslope is larger. A clock CLK starts at the same time that the referencevoltage Vslope starts to transition, and a counter 514 counts up incorrespondence with the clock CLK in the case where the output of thecomparator 513 is high level, and stops the signal of the count at thesame time when the output of the comparator 513 is reversed to lowlevel. The count value at this time is held in one of a memory 516 and amemory 517 as a digital signal.

The memory 516 holds a digital signal obtained by A/D converting thesignal (hereinafter, “N signal”) of the reset level of the FD 504, andthe memory 517 holds a digital signal obtained by A/D converting asignal (hereinafter, “S signal”) obtained by superimposing the signal ofthe photoelectric conversion portion 201A or the photoelectricconversion portion 201B on the N signal of the FD 504. Whether the countvalue of the counter 514 is written to the memory 516 or 517 is assignedby the switch 515. The difference between the signals held in thememories 516 and 517 is calculated by subtracting the N signal from theS signal with a CDS circuit 518. This difference is then output to thesignal processor 105 via a digital signal output line 519, under thecontrol of the drive circuit portion 402.

Note that one readout circuit 509 is disposed per column of pixels, andpixel signals are read out in units of rows. In this case, the selectionswitch 506 is controlled in units of rows, and the pixel signals of theselected row are output at the same time to respective column outputlines 507. The pixel signals of the pixel portion 401 can be read out tothe signal processor 105 faster as the number of readout circuits 509increases.

FIG. 6 is a timing chart showing an example of an operation for readingout charge from the unit pixels 200 of the image sensor 100 having thecircuit configuration shown in FIG. 5 . The timing of each drive pulse,the reference voltage Vslope, the clock CLK, and the horizontal scanningsignal are schematically shown. A potential V1 of the column output line507 at each timing is also shown.

Prior to reading out the signal from the photoelectric conversionportion 201A, the signal line PRES of the reset switch 503 changes to Hi(t600). The gate of the SF (source follower amplifier) 505 is therebyreset to a reset power supply voltage. The control signal PSEL is set toHi at time t601, and the SF 505 enters an operating state. Resetting ofthe FD 504 is canceled by setting the control signal PRES to Lo at t602.The potential of the FD 504 at this time is output to the column outputline 507 as a reset signal level (N signal), and input to the readoutcircuit 509.

By setting the control signal PSH to Hi and Lo at times t603 and t604 toturn the switch 511 on and off, the N signal output by the column outputline 507 is held in the capacitor 512 after being amplified by a desiredgain in the amplifier 510. The potential of the N signal held in thecapacitor 512 is input to one input of the comparator 513. After theswitch 511 is turned off at time t604, the reference voltage Vslope isdecreased over time from an initial value by a slope voltage generationcircuit (not shown) from time t605 to t607. The clock CLK is supplied tothe counter 514 together with the reference voltage Vslope starting totransition. The value of the counter 514 increases according to thenumber of CLKs. Then, when the reference voltage Vslope input to thecomparator 513 reaches the same level as the N signal, an output COMP ofthe comparator 513 changes to low level, and operation of the counter514 also stops at the same time (time t606). The value of the counter514 at the time that operation stops will be the A/D converted value ofthe N signal. The counter 514 is then connected to the memory 516 by theswitch 515, and the digital value of the N signal is held in the Nsignal memory 516.

Next, the photoelectric charge accumulated in the photoelectricconversion portion 201A is transferred the FD 504 by setting the controlsignal PTXA to Hi and then Lo at times t607 and t608 after holding thedigitized N signal in the N signal memory 516. Then, the change in thepotential of the FD 504 that depends on the amount of charge is outputto the column output line 507 as the signal level (opticalcomponent+reset noise component (N signal)), and input to the readoutcircuit 509. The input signal (S(A)+N), after being amplified by adesired gain in the amplifier 510, is held in the capacitor 512 at thetiming at which the switch 511 is turned on and off by the controlsignal PSH being set to Hi and then Lo at times t609 and t610. Thepotential held in the capacitor 512 is input to one input of thecomparator 513. After the switch 511 is turned off at time t610, thereference voltage Vslope is decreased over time from the initial valueby the slope voltage generation circuit from time t611 to t613. CLK issupplied to the counter 514 together with the reference voltage Vslopestarting to transition. The value of the counter 514 increases accordingto the number of CLKs. Then, when the reference voltage Vslope input tothe comparator 513 reaches the same level as the S signal, the outputCOMP of the comparator 513 changes to low level, and the operation ofthe counter 514 also stops at the same time (time t612). The value ofthe counter 514 at the time that operation stops is the A/D convertedvalue of the S(A)+N signal. The memory 517 is then connected to thecounter 514 by the switch 515, and the digital value of the S(A)+Nsignal is held in the S signal memory 517. A differential signal level(optical component) is calculated by the CDS circuit 518 from thesignals held in the memory 516 and the memory 517, and an S(A) signalfrom which the reset noise component has been removed is acquired. TheS(A) signal is sequentially sent to the signal processor 105 under thecontrol of the controller 104.

Operations for reading out signals from the photoelectric conversionportion 201A of the unit pixel 200 have been described above. In thecase of reading out signals from the other photoelectric conversionportion 201B of the unit pixel 200, driving need only be similarlyperformed in accordance with the timing chart of FIG. 6 . In this case,however, the control signal PTXB is set to Hi and then Lo at times t607and t608, instead of the control signal PTXA. That is, by setting thecontrol signal PTXA to Hi and Lo to output the pixel signal S(A) whendriving from time t600 to time t613 in FIG. 6 is performed the firsttime, and then setting the control signal PTXB to Hi and Lo to outputthe pixel signal S(B) when driving from time t600 to time t613 in FIG. 6is performed the second time, output of one row of image signals iscompleted. By repeating this for all of the rows, output of the pixelsignals S(A) and S(B) of all the pixels is completed.

FIG. 7 is a diagram showing an exemplary configuration of the signalprocessor 105 and the output portion 106 of the image sensor 100. Thesignal processor 105 has an image capture signal processor 701 and afocus detection signal processor 702. The pixel signals output from thereadout portion 103 are input to the signal processor 105 via thedigital signal output line 519. The input signals are processed inaccordance with control from the controller 104. Note that the imagecapture signal processor 701 and the focus detection signal processor702 are each assumed to be provided with a memory (not shown).

In the image capture signal processor 701, an image capture signal iscalculated from the signals output from the readout portion 103. Thatis, the image capture signal processor 701 receives the pixel signalsS(A) and S(B) of the photoelectric conversion portion 201A and thephotoelectric conversion portion 201B of the unit pixel 200 and performscombination processing to calculate an S(A+B) signal. The image capturesignal processor 701 then sends the pixel signal S(A+B) to the outputportion 106 via an image capture signal output line 703. The imagecapture signal processor 701 is able to reduce the amount of signaltransmission to outside of the image sensor 100, by combining the pixelsignals S(A) and S(B) of the two photoelectric conversion portions andoutputting the combined signal to outside of the image sensor 100 fromthe output portion 106. Note that operation of the pixel signal S(A) andthe pixel signal S(B) becomes possible at the stage at which both pixelsignals of the unit pixel are brought together. The pixel signal S(A)read out first is held in memory, and when the pixel signal S(B) is readout and input to the image capture signal processor 701, the operationS(A)+S(B) is sequentially performed, and the resultant signal is outputfrom the output portion 106.

Note that the image capture signal processor 701 may further performprocessing such as combining and averaging the signals of the unitpixels 200. For example, in a pixel portion having a typicalconfiguration in which red (R), green (G) and blue (B) color filters areprovided in a Bayer array, the signal transmission amount can be furtherreduced if the signals of adjacent pixels of the same color are combinedand averaged before being output to the output portion 106. Also, ratherthan output the signals of the entire pixel portion to the outputportion 106, the signals of only a required area may be output. Thisprocessing is controlled by the controller 104.

Next, processing by the focus detection signal processor 702 will bedescribed. The focus detection signal processor 702 calculates a focusdetection signal from the signals output from the readout portion 103,and outputs the calculated focus detection signal. In order to performphase difference detection, the pixel signal S(A) and the pixel signalS(B) are both required, as described above. However, the signaltransmission amount is huge when the pixel signals S(A) and the pixelsignals S(B) of all the pixels of the pixel portion 401 are output tooutside of the image sensor 100 from the output portion 106, and thisserves as an impediment to high-speed readout.

In view of this, computational operation processing is performed in thefocus detection signal processor 702, and a reduced amount of signalsare output from the output portion 106. For example, luminance values Yare calculated by respectively performing Bayer addition on the pixelsignals S(A) and S(B), and luminance signals Y(A) and Y(B) are output.The computational operations for focus detection may also be performedafter converting the signals into Y values, and the signal transmissionamount can be reduced to one quarter, by converting the signals into Yvalues before being output from the image sensor 100. Note that sincesignals in Bayer units are required in calculating the Y values, thepixel signals input to the focus detection signal processor 702 are heldin memory until the signals required for calculation are broughttogether. In other words, because the signals of the G and B rows areoutput after the signals of the R and G rows have been output, the pixelsignals S(A) and the pixel signals S(B) of the R and G rows are held inmemory, and when the signals of the G and B rows are output,computational operations are sequentially performed on the luminancesignals Y(A) and Y(B), and the resultant signals are output from theoutput portion 106 via a signal line 704.

Also, the focus detection signal processor 702 may further perform acorrelation operation, and output the resultant value to the outputportion 106 through the signal line 704. Note that phase differencedetection using a correlation operation can be implemented by awell-known technique. In the case of outputting only correlationoperation values, the amount of signals that are output can be greatlyreduced, although this is dependent on the number of divided areas atthe time of the correlation operation.

As described above, signal processing for outputting only requiredsignals to outside of the image sensor 100 is performed in the imagesensor 100 that is provided with the image capture signal processor 701and the focus detection signal processor 702. The signal transmissionamount can thereby be reduced, and image capture data and focusdetection information can both be obtained at high speed.

Note that, in the present embodiment, a configuration was adopted inwhich a memory is provided in both the image capture signal processor701 and the focus detection signal processor 702. However, aconfiguration may be adopted in which the memories are provided upstreamthereof, and signals are sent to the image capture signal processor 701and the focus detection signal processor 702 at the stage at which thesignals required in the computational operations in each processingportion are brought together.

Also, in the description of the present embodiment, the luminancesignals Y(A) and Y(B) are output as focus detection signals, but aconfiguration in which only the luminance signal Y(A) is output from theoutput portion 106 may be adopted. Specifically, an image capturesignal, that is, the pixel signal S(A+B), is output from the imagecapture signal processor 701. Thus, the focus detection signal may beobtained by calculating the luminance signal Y(A+B) from the pixelsignal S(A+B) in the phase difference detection portion 108 or the likeafter the pixel signal S(A+B) has been output to outside of the imagesensor 100, and subtracting the luminance signal Y(A) to calculate theluminance signal Y(B). The signal transmission amount can thus befurther reduced, by outputting only the luminance signal Y(A) from thefocus detection signal processor 702.

For example, if a signal processor is not provided, in the case wherethe pixel count is 20 megapixels, pixel signals S(A) and S(B) for allthe pixels, that is, 40 million pieces of data, need to be output. Onthe other hand, in the case where the Y value is calculated and outputas the focus detection signal by an image sensor that is provided withthe signal processor of the present embodiment, 20 million pieces ofdata for image capture and 5 million pieces of data for focus detection(20 million/4) will be output, evidently reducing the signaltransmission amount. As a result, high-speed readout becomes possible.Also, in the case where the focus detection signal is a correlationoperation value, the signal transmission amount is clearly furtherreduced.

In the timing chart of FIG. 6 , it is also possible to obtain signals ofthe unit pixels 200 in which the charge of the photoelectric conversionportion 201A and the charge of the photoelectric conversion portion 201Bare combined by simultaneously controlling the control signals PTXA andPTXB at times t607 and t608. Specifically, it is possible to obtain thepixel signal S(A+B), if signal readout is performed by controlling thecontrol signals PTXA and PTXB so as to change to Hi and Lo at the sametime, after reading out the signal of the photoelectric conversionportion 201A, in accordance with the timing chart of FIG. 6 . In thiscase, since readout of the reset signal decreases by one, even fasterreadout becomes possible.

In the case where the pixel signals S(A) and S(A+B) are read out fromthe pixels, the pixel signal S(B) can be obtained, if processing forsubtracting the pixel signal S(A) from the pixel signal S(A+B) isperformed in the focus detection signal processor 702. Alternatively,the focus detection processing portion 702 may process and output onlythe pixel signal S(A), and the pixel signal S(B) or the luminance signalY(B) may be calculated in the phase difference detection portion 108.

Second Embodiment

Next, a second embodiment of the present invention will be described. Inthe first embodiment, output of focus detection signals was performed inthe entire area of the pixel portion, but if the signals of onlyrequired areas are selected and output with regard to the focusdetection signals, a further increase in processing speed can berealized.

FIGS. 8A and 8B are diagrams showing an exemplary output area of focusdetection signals in a pixel area. Focus detection signals and imagecapture signals are output from the shaded areas, and only image capturesignals are output from the other areas. For example, focus detectionsignals of only target areas are discretely (selectively) output over awide range of pixels, as in the example shown in FIG. 8A. It therebybecomes possible to obtain focus detection information on the entirepixel area, while suppressing the amount of signals that are output tooutside of the image sensor 100. In the case of the example shown inFIG. 8B, it becomes possible to obtain detailed focus detectioninformation about a partial area, and also to suppress the amount ofsignals that are output to outside of the image sensor 100. Selection ofthese areas in which to perform output is controlled by the controller104. The focus detection signal processor 702 reads out only signals ofthe output target areas from memory and performs computationaloperations on the read signals.

Note that the output of the focus detection signals in partial areassuch as is shown in FIGS. 8A and 8B may be Y value signals orcorrelation operation results, but may also be pixel signals S(A).Although the amount of signals that are transmitted increases comparedwith Y value signals or correlation operation results, suppression ofthe signal transmission amount is achieved, since output is performedfrom only required areas. Also, this is realizable even with acomparatively small-scale signal processing circuit.

Note that processing for combining and averaging focus detection signalsmay also be performed in the focus detection signal processor 702. Inthis case, processing for combining and averaging is performed on pixelsignals S(A) and on pixel signals S(B).

As described above, signal processing for outputting only requiredsignals to outside of the image sensor 100 is performed in the imagesensor 100 that is provided with the image capture signal processor 701and the focus detection signal processor 702. The transmission amount offocus detection signals can thereby be reduced, and image capture dataand focus detection information can both be obtained at high speed andefficiently.

Note that there is a method of shortening the readout time of one frame,by thus limiting the pixels to be used in focus detection. Normally, anincrease in the readout time is suppressed by respectively outputtingthe signals of two photoelectric conversion portions within the unitpixels in only rows that are used in focus detection processing, andcombining the signals of the two photoelectric conversion portions andoutputting only image generation signals in rows that are not used infocus detection processing. In this case, it is possible to combine theindividual output signals of the two photoelectric conversion portionsoutput as focus detection signals and use the combined signals as pixelimage capture signals. However, a problem arises in that a differenceoccurs in noise level and the like due to the signal readout method andthe method of combining the output signals of the two photoelectricconversion portions differing depending on whether or not the row willbe used in focus detection processing, resulting in a deterioration inthe captured image that is obtained. However, by providing a focusdetection signal processor as in the present embodiment, the signalsfrom the pixel portion are all read out at a similar readout timing, andpixels to be output by the focus detection signal processor 702 can beselected. Thus, the noise amount of the pixel signals S(A+B) that areused in image capture does not change depending on the area, enablinghigh quality captured images to be obtained.

Third Embodiment

Next, a third embodiment of the present invention will be described. Inthe second embodiment, an example was described in which the focusdetection signal processor 702 selects and outputs only signals of arequired output target area among the focus detection signals. In thepresent embodiment, in further pursuit of this approach, an example willbe described in which the required area among the focus detectionsignals is set based on a user input or the area of a subject detectedby a subject detection portion.

The configuration of an image capturing apparatus of the presentembodiment is, as shown in FIG. 9 , the configuration of the imagecapturing apparatus of the first and second embodiments shown in FIG. 1to which a subject detection portion 105 a has been added. The remainingconfiguration is similar to the configuration of FIG. 1 , and thusdescription of portions that are the same is omitted and only portionsthat are different will be described.

In FIG. 9 , the subject detection portion 105 a, on receipt of digitalsignal output for image generation from the readout portion 103, detectsa subject using a well-known pattern recognition processing circuit, anddetermines a focus detection area for performing focus detectionprocessing. A physical feature such as the face or eyes of a person oran animal, for example, are given here as a subject that is detected.Also, the subject detection portion 105 a may be provided internallywith a memory that temporarily stores image generation signals, in orderto perform subject detection processing.

The signal processor 105 has an image capture signal processor 701 and afocus detection signal processor 702, as already described using FIG. 7in the first embodiment. The operations of the image capture signalprocessor 701 are similar to the first embodiment.

On the other hand, in the focus detection signal processor 702,similarly to the second embodiment, a required area among of the digitalsignal output for focus detection is selected and output to the outputportion 106. In the case where the focus detection area is set manually,however, the focus detection signal processor 702 selectively outputsfocus detection signals of a focus detection area arbitrarily selectedby the user. Alternatively, in the case where the focus detection areais set automatically, the focus detection signal processor 702, onreceipt of a subject detection result of the subject detection portion105 a, selectively outputs focus detection signals of the area in whicha subject is detected. The output portion 106 outputs digital signalsfor image generation received from the image capture signal processor701 and digital signals for focus detection received from the focusdetection signal processor 702 to outside of the image sensor 100.

The phase difference detection portion 108, on receipt of a digitalsignal for focus detection from the output portion 106, calculates aphase difference evaluation value for performing focus detectionemploying a phase difference detection method. In the presentembodiment, the focus detection signal that is input to the phasedifference detection portion 108 is a signal that the focus detectionsignal processor 702 within the signal processor 105 provided inside theimage sensor 100 outputs after selecting an area. Accordingly, becausethe focus detection signal that the output portion 106 of the imagesensor 100 transmits to the phase difference detection portion 108 isonly a signal required for focus detection control, the transmissionband is efficiently used. Furthermore computational operation processingfor phase difference evaluation value calculation in areas that will notbe required in focus detection control and processing for extractingsignals that will ultimately be required in focus detection control arealso not required in the internal processing of the phase differencedetection portion 108. Accordingly, the phase difference detectionportion 108 is able to increase the processing speed at which the phasedifference evaluation value is calculated. Furthermore, the scale ofprocessing circuitry of the phase difference detection portion 108 canalso be reduced.

Note that the display unit 110 in FIG. 9 is used not only to displayimage signals received from the overall control/operation unit 109 butalso to display focus detection areas that the user of the imagecapturing apparatus is able to arbitrarily select. The display unit 110is also used in order to display the subject area, which is the area inwhich the subject detected by the subject detection portion 105 aexists. Also, the operation unit 113 is used for various types ofinputs, but is used also in order for the user of the image capturingapparatus to set an arbitrary focus detection area. If the display unit110 is a touch panel, however, input operations of the operation unit113 may be substituted for touch operations on the display unit 110.

Next, FIG. 10 is a flowchart showing the flow of processing for imagecapture plane phase difference AF in the present embodiment. Whenprocessing for image capture plane phase difference AF is started,first, in step S401, the pixel portion 401 of the image sensor 100 isdriven, and the signals (focus detection signals) of the plurality ofPDs that are included in the unit pixels 200 of the entire area of thepixel portion 401 are individually read. At this time, the drive circuitportion 402 may drive the pixel portion 401 with a readout scanningmethod such as row thinning readout, row addition readout, row portionreadout, column thinning readout, column addition readout, and columnportion readout, according to a required frame rate. As aforementionedabove, however, the drive circuit portion 402 drives the pixel portion401 uniformly under the same conditions, and thus problems such ascrosstalk and blooming tending to occur do not arise.

Thereafter, the signal of each PD undergoes A/D conversion in thereadout portion 103, and a digital signal for focus detection isobtained. Furthermore, the readout portion 103 is also able to generateimage generation signals by combining the digital signals of theplurality of PDs for every unit pixel 200.

The focus detection signals and the image generation signals of theentire area of the pixel portion 401 are output to the signal processor105 from the readout portion 103. The image generation signals areoutput to outside of the image sensor 100 from the output portion 106via the signal processor 105, and are processed by the image processor107. Thereafter, a generated image is displayed on the display unit 110by the overall control/operation unit 109. A moving image iscontinuously displayed on the display unit 110, as a result of imagegeneration signals being continuously output from the image sensor 100at a predetermined frame rate.

Next, in step S402, the mode for selecting a focus detection area isconfirmed. Here, in the case where the focus detection area has beenmanually set in advance, in order for the user to focus the taking lens101 on an arbitrary area, the processing transitions to step S403. Anexemplary operation screen that is displayed on the display unit 110when the user selects a focus detection area in this case is shown inFIG. 11 . As shown in FIG. 11 , a plurality of focus detection areas1101 that can be selected are displayed on the operation screen. Here, a7:5 (width:height) detection frame is shown, but the detection frame maybe divided into smaller areas or larger areas. Also, rather thanselecting from a predetermined detection frame, the user may designatean arbitrary position from the entire shot screen. The user selects anarea 1102 that includes a person's face, for example, as the area onwhich to focus. Position information on the focus detection area 1102selected by the user is input to the signal processor 105 from theoperation unit 113, via the overall control/operation unit 109.

Next, in step S403, the focus detection signal processor 702 within thesignal processor 105 selects the focus detection signals of the areadesignated by the user, among the focus detection signals of the entirearea of the pixel portion 401 that are output to the signal processor105 from the readout portion 103 at step S401. The focus detectionsignals selected by the focus detection signal processor 702 are inputto the phase difference detection portion 108, via the output portion106. At this time, in the present embodiment, high-speed transmission ispossible, because the focus detection signals that are transmitted tothe phase difference detection portion 108 by the output portion 106 areonly signals required for focus detection control, that is, only focusdetection signals of the area designated by the user.

Next, at step S404, the phase difference detection portion 108, onreceipt of the digital signals for focus detection from the outputportion 106, calculates a phase difference evaluation value forperforming focus detection employing a phase difference detectionmethod. At this time, the phase difference detection portion 108 is notrequired to perform computational operation processing for phasedifference evaluation value calculation in areas that will not berequired in focus detection control or processing for extracting signalsthat will ultimately be required in focus detection control.Accordingly, focus detection control can be performed at high speed.

Next, in step S405, based on the phase difference evaluation valuecalculated by the phase difference detection portion 108, the lenscontroller 114 calculates optical system drive information and controlsthe focus lens position of the taking lens 101.

Next, in step S406, it is checked whether the image capturing apparatusshould end shooting. If a shooting end operation is input by the userfrom the operation unit 113, shooting is directly ended. If a shootingend operation is not input by the user from the operation unit 113, theprocessing transitions to step S401, and shooting and image captureplane phase difference AF processing are continued.

On the other hand, in the case where, in step S402, settings have beenconfigured such that the area on which to focus the taking lens 101 isautomatically determined by the image capturing apparatus, theprocessing transitions to step S407. In step S407, the subject detectionportion 105 a, on receipt of the image generation signal from thereadout portion 103, performs subject detection processing forautomatically determining the area on which to focus the taking lens101. The subject to be detected is, for example, the face or eyes of aperson or an animal. Various well-known pattern recognition processingcan be applied as the method of subject detection processing. Atechnique called template matching or deep learning, for example, isgiven as a typical pattern recognition technique.

An exemplary screen that is displayed on the display unit 110, in orderto show the detected subject area detected by the subject detectionportion 105 a is shown in FIG. 12A. As shown in FIG. 12A, a subject area1201 detected by the subject detection portion 105 a as an area thatincludes a person's face is displayed. The subject detection portion 105a outputs horizontal/vertical address information of the detectedsubject area 1201 within the image to the signal processor 105. Thesignal processor 105 outputs horizontal/vertical address information ofthe subject area 1201 within the image to the overall control/operationunit 109 via the output portion 106, in order to display thehorizontal/vertical address information as a subject area on the displayunit 110. The overall control/operation unit 109 composes the subjectarea information with the generated image processed by the imageprocessor 107, and displays the composite image on the display unit 110.Also, as will be discussed in detail later, the signal processor 105uses the horizontal/vertical address information of the subject area1201 within the image in order to select a focus detection signal.

Also, the case where a person's face is framed in close-up is shown inFIG. 12B as another example of a subject detection result. In an examplesuch as FIG. 12B, the area including the person's face is large, andthus if the focus detection signals in an area over a wide range thatincludes the entire face were output to the downstream phase differencedetection portion 108, the communication time would be long, and thiswould serve as an impediment to high-speed focus detection control.Accordingly, as shown in FIG. 12B, in the case where a person's face isdetected in close-up, preferably the subject detection portion 105 afurther extracts a characteristic portion (e.g., eyes) of the face, andsets the extracted portion as a subject detection area 1202. This isrealized, for example, by detecting the case where the pixel count ofthe area in which the subject detection portion 105 a detected the faceexceeds a predetermined pixel count, and performing control so as toswitch the subject detection target.

Next, in step S408, the detection result of the subject detectionportion 105 a in step S407 is confirmed. If a subject was detected instep S407, the processing transitions to step S409.

In step S409, the signal processor 105 selects the focus detectionsignals of the area detected by the subject detection portion 105 a,among the focus detection signals read out from the entire area of thepixel portion 401. The focus detection signals selectively output fromthe signal processor 105 are input to the phase difference detectionportion 108 via the output portion 106. In this case, in the presentembodiment, the focus detection signals that are transmitted to thephase difference detection portion 108 by the output portion 106 areonly signals of the area required for focus detection control, and canthus be transmitted at high speed.

Next, in step S404, the phase difference detection portion 108, onreceipt of the digital signals for focus detection from the outputportion 106, calculates a phase difference evaluation value forperforming focus detection employing a phase difference detectionmethod. In this case, the phase difference detection portion 108 is notrequired to perform computational operation processing for phasedifference evaluation value calculation in areas that will not berequired in focus detection control or processing for extracting signalsthat will ultimately be required in focus detection control.Accordingly, focus detection control can be performed at high speed.Subsequently, processing that has already been described is executed insteps S405 and S406.

On the other hand, if it is confirmed in step S408 that a subject wasnot detected in step S407, the processing transitions to step S410.

In step S410, the lens controller 114 performs control so as to searchdrive the focus lens of the taking lens 101 by a predetermined amount.Furthermore, in step S411, the focus lens position of the taking lens101 is checked, and it is determined whether the search drive has ended.Here, if the search drive is in progress, the processing transitions tostep S401. Accordingly, if the setting in step S402 is to automaticallydetermine the focus detection area, and a state in which the subjectdetection portion 105 a is not able to detect a subject in step S407continues, the search drive of the focus lens is continued. If, in stepS411, the search drive from the infinity end to the close end has ended,however, the processing transitions to step S412.

In step S412, processing in the case where the subject detection portion105 a is not able to detect a subject in step S407, even when the lenscontroller 114 repeats the operation for search driving the focus lensin step S410, is performed. Here, in order to provisionally determinethe focus lens position of the taking lens 101, the signal processor 105selects the focus detection signals of a provisional area, among thefocus detection signals read out from the entire area of the pixelportion 401. The focus detection signals selectively output from thesignal processor 105 are input to the phase difference detection portion108, via the output portion 106.

Next, in step S404, the phase difference detection portion 108, onreceipt of the digital signals for focus detection from the outputportion 106, calculates a phase difference evaluation value forperforming focus detection employing a phase difference detectionmethod. Subsequently, processing that has already been described isexecuted in steps S405 and S406.

As described above, in the present embodiment, the focus detectionsignals that are transmitted to the phase difference detection portion108 by the output portion 106 of the image sensor 100 are only signalsrequired for focus detection control, and thus high-speed transmissionis possible. The phase difference detection portion 108 is not requiredto perform computational operation processing for phase differenceevaluation value calculation in areas that will not be required in focusdetection control or processing for extracting signals that willultimately be required in focus detection control. Accordingly, thephase difference detection portion 108 is able to increase theprocessing speed at which the phase difference evaluation value iscalculated. Therefore, focus detection control by image capture planephase difference AF can be performed at high speed.

Fourth Embodiment

Next, a fourth embodiment of the present invention will be described. Inthe fourth embodiment, the configuration of the unit pixels 200 of thepixel portion 402 is different. FIG. 13 is a diagram in which the lightreceiving portion 102 and micro lens array of the image sensor 100 areobserved from the optical axis direction (Z direction). Fourphotoelectric conversion portions 901A, 901B, 901C and 901D are disposedwith respect to one micro lens 202. By thus having a total of fourphotoelectric conversion portions composed of two each in the X-axisdirection and the Y-axis direction, the light that has passed through anexit pupil area divided in four can be respectively received. The signalreadout method and the processing by the signal processor 105 in animage sensor 100 that is constituted by a pixel portion 401 in whichsuch unit pixels are each provided with four photoelectric conversionportions will be described.

FIG. 14 is a schematic diagram showing an example of the configurationsof a unit pixel 900 and the readout portion 103. The configuration inFIG. 14 is provided with readout circuits each corresponding to adifferent one of photoelectric conversion portions. That is, the pixelsignal of a photoelectric conversion portion 901A is output to a readoutcircuit 1001A. Hereinafter, similarly, the pixel signals ofphotoelectric conversion portions 901B to 901D are respectively outputto readout circuits 1001B to 1001D. Since the signal readout operationfrom the photoelectric conversion portions can be implemented with amethod that is substantially similar to the drive method described inFIGS. 5 and 6 , description thereof is omitted.

Processing by the signal processor 105 on the signals read out from eachphotoelectric conversion portion will be described. The image capturesignal processor 701 calculates image capture signals from signals thathave been read out. That is, the image capture signal processor 701, onreceipt of pixel signals S(A), S(B), S(C) and S(D) of the plurality ofphotoelectric conversion portions 901A, 901B, 901C and 901D of the unitpixel 900, performs combination processing and calculates a pixel signalS(A+B+C+D). The pixel signal S(A+B+C+D) is then sent to the outputportion 106 via the image capture signal output line 703. The imagecapture signal processor 701 is able to reduce the amount of signaltransmission to outside of the image sensor 100, by combining thesignals of the plurality of photoelectric conversion portions andoutputting the combined signals to outside of the image sensor 100 fromthe output portion 106. In an image sensor that is provided with fourphotoelectric conversion portions, as with the unit pixel 900, thiseffect is further enhanced.

Next, processing by the focus detection signal processor 702 will bedescribed. In the case of an image sensor that is provided with fourphotoelectric conversion portions per unit pixel 900 as shown in FIG. 13, the signals of the four photoelectric conversion portions need to beindividually output in order to perform focus detection, and thus thesignal transmission amount is considerable, which is not desirable interms of high-speed readout. As described in the first embodiment, it ispreferable to calculate and output Y values or to output onlycorrelation operation results. If the signals of only required areas areoutput, the signal transmission amount can be further reduced.

In the case of a configuration in which the unit pixel 900 is providedwith 2×2 photoelectric conversion portions as shown in FIG. 13 , phasedifference detection can also be performed in the up-down direction, inaddition to the right-left direction. For example, the pixel signalsS(A) and S(C) are combined and output, and the pixel signals S(B) andS(D) are combined and output. In this case, focus detection employing aphase difference method in which pupil division is performed in theright-left direction from obtained focus detection signals becomespossible. Also, in the case the pixel signals S(A) and S(B) are combinedand output and the pixel signals S(C) and S(D) are combined and output,focus detection employing a phase difference method in which pupildivision is performed in the up-down direction from obtained focusdetection signals becomes possible. By switching and outputting thesesignals according to the subject, accurate focus detection can beperformed on respective subjects in vertical bands and horizontal bands.Also, the output pattern may be changed depending on the pixel area.

Furthermore, the focus detection signal processor 702 may perform acorrelation operation using the combined signals that are obtained(e.g., S(A+C) and S(B+D)), and output only the result thereof. Ifcorrelation operation processing is performed in the focus detectionsignal processor 702, the signal transmission amount that is output fromthe image sensor 100 can be reduced. Also, the signal transmissionamount can be suppressed, even when correlation operations in both theleft-right direction and the up-down direction are performed in the samearea and the results thereof are output.

In an image sensor provided with multi-division pixels, by thusproviding the focus detection signal processor 702 within the imagesensor, an increase in the amount of signals that are output from theimage sensor can be suppressed, and image capture data and focusdetection information can be obtained at high speed. Furthermore, sincephase difference information can be acquired in the right-left directionand the up-down direction, focus detection can be accuracy performed.Although, in the present embodiment, an image sensor having fourphotoelectric conversion portions per unit pixel was described as anexample, a configuration having more photoelectric conversion portionsmay be adopted. In order to add more parallax, only required PD signalsmay be output, or signals may be diagonally combined and output.

Fifth Embodiment

Since the signal processor 105 of an image sensor 100 such as describedin the first to third embodiments has large-scale circuitry, there is ahigh possibility that the area of the image sensor 100 will be large. Inview of this, in the present embodiment, the configuration of an imagesensor 100 that suppresses an increase in area will be described.

FIGS. 15 and 16 are configuration diagrams of an image sensor 100 in thefifth embodiment. The image sensor of the present embodiment has aconfiguration (multilayer structure) in which a pixel area chip 1501 anda signal processing chip 1502 are stacked. The wirings betweensemiconductor chips (semiconductor substrates) are electricallyconnected using micro bumps or the like, by a well-known substratestacking technique.

A pixel area chip 1501 is provided with a pixel portion 401 in whichunit pixels 200 each provided with a plurality of photoelectricconversion portions are arrayed in a matrix, a drive circuit portion402, and a readout portion 103. The drive circuit portion 402 sendsdrive signals to the pixels of the pixel portion 401. Note that, in FIG.15 , the unit pixels 200 have two photoelectric conversion portions, butthe number of photoelectric conversion portions is not limited thereto.

The readout portion 103 is configured to be provided with one readoutcircuit 509 per pixel column, for example, and reads out pixel signalsof the pixel portion 401. Vertical and horizontal selection of readpixel signals is performed under the control of the drive circuitportion 402, and selected signals are sequentially transferred to thesignal processor 105.

A signal processing chip 1502 is provided with a controller 104, asignal processor 105, and an output portion 106. The signal processor105 has an image capture signal processor 701, and a focus detectionsignal processor 702, and functions to process pixel signals read outfrom the readout portion 103 and output the processed pixel signals tooutside of the image sensor 100 via the output portion 106. Since thesignal processing in the signal processor 105 is similar to theprocessing described in the first to third embodiments, descriptionthereof is omitted. The image sensor 100 has a configuration in whichthe pixel area chip 1501 and the signal processing chip 1502 areintegrally formed by being stacked, as shown in FIG. 16 .

As described above, by providing the image sensor with a stackedstructure, sufficient area for the signal processor 105 can be secured,enabling large-scale circuitry to be mounted. In the signal processor105, the signal transmission amount can be reduced by performing signalprocessing for outputting only required signals to outside of the imagesensor 100, and it becomes possible to obtain both image capture dataand focus detection information at high speed.

Sixth Embodiment

In an image sensor constituted by a pixel portion that is provided witha plurality of photoelectric conversion portions per unit pixel such asdescribed in the first to third embodiments, a large number of thereadout circuits 509 is preferable. For example, since pixel signals canbe output at the same time for all pixels and A/D converted on apixel-by-pixel basis in the case of adopting a configuration providedwith one readout circuit per unit pixel, and further adopting aconfiguration provided with one readout circuit per photoelectricconversion portion, faster readout becomes possible. In this case, areais required in order to dispose the readout circuits, and thus an imagesensor having a stacked structure is desirable.

FIG. 17 is a diagram showing the configuration of an image sensor 100 inthe sixth embodiment. The image sensor of the present embodiment has aconfiguration in which a pixel area chip 1301, a readout circuit chip1302 and a signal processing chip 1303 are stacked. The wirings betweenthe semiconductor chips (semiconductor substrates) are electricallyconnected using micro bumps or the like, by a well-known substratestacking technique.

The pixel area chip 1301 is provided with a pixel portion 401 in whichunit pixels 200 provided with a plurality of photoelectric conversionportions are arrayed in a matrix and a drive circuit portion 402. Thedrive circuit portion 402 sends drive signals to the pixels of the pixelportion 401. Note that, in FIG. 17 , the unit pixels 200 have twophotoelectric conversion portions, but the number of photoelectricconversion portions is not limited thereto.

The readout circuit chip 1302 is provided with a readout portion 103, avertical selection circuit 1304, and a horizontal selection circuit1305. The readout portion 103 has a large number of readout circuits 509corresponding one-to-one with the unit pixels or the photoelectricconversion portions, and the pixel signals of the pixel portion 401 areoutput thereto. The pixel signals output to the readout circuit 509 aretransferred to the signal processor 105 sequentially under the controlof the vertical selection circuit 1304 and the horizontal selectioncircuit 1305.

The signal processing chip 1303 is provided with a controller 104, asignal processor 105, and an output portion 106. The signal processor105 has an image capture signal processor 701 and a focus detectionsignal processor 702, and functions to process pixel signals read out bythe readout portion 103 and output the processed pixel signals tooutside of the image sensor 100 via the output portion 106. Since thesignal processing in the signal processor 105 is similar to theprocessing described in the first to third embodiments, descriptionthereof is omitted.

The image sensor 100 has a configuration in which the pixel area chip1301, the readout circuit chip 1302 and the signal processing chip 1303are integrally formed by being stacked. Incidentally, in the case ofadopting a configuration in which one readout circuit is provided perphotoelectric conversion portion, as with the configuration of thepresent embodiment, the time for the pixel signals to be output to thesignal processor 105 is significantly reduced. For example, when thetime taken for signal output by one photoelectric conversion portion isgiven as α, the time taken in order to read out the pixel signals of oneframe will be α×number of rows, in the case of an image sensor that isprovided with one readout circuit per column. On the other hand, in thecase where one readout circuit is provided per photoelectric conversionportion, the pixel signals of one frame can be read out in a time of α.In this case, however, there is concern that the receipt of pixelsignals and the signal processing in the signal processor 105 willdetermine the readout speed. However since the signal processor 105 ofthe image sensor 100 having a stacked structure, as in the presentembodiment, can be arranged to have a large area, a large number oftransmission lines of signals from the readout circuit chip 1302 can beprovided, enabling pixel signals to be sent to the signal processor 105at high speed. Also, since a large number of signal processing circuitscan be mounted in the signal processor 105, parallel processing becomespossible, and signal processing time can also be reduced.

As described above, in the case where an image sensor has a stackedstructure, sufficient area can be taken for the readout portion and thesignal processor. As a result of the high-speed signal readout from thepixel portion of the image sensor of the present embodiment, and thesignal processor 105 performing signal processing for outputting onlyrequired signals to outside of the image sensor, the signal transmissionamount can be reduced, and both image capture data and focus detectioninformation can be obtained at high speed.

Seventh Embodiment

FIG. 18 is a block diagram showing the configuration of an imagecapturing apparatus of a seventh embodiment of the present invention. InFIG. 18 , constituent portions that are the same as the third embodimentwill be given the same reference signs, and a detailed description isomitted.

In the seventh embodiment, the image capturing apparatus, when viewed asa whole, consists of the same constituent elements as the thirdembodiment, but the constituent elements internally included in an imagesensor 800 differs from the third embodiment. The image sensor 800 ofthe seventh embodiment is configured to include a light receivingportion 801, a readout portion 802, a signal processor 803, an imageprocessor 804, a subject detection portion 805, a phase differencedetection portion 806, an output portion 807, and a controller 104.

The light receiving portion 801 receives the light of an optical imageformed by the taking lens 101. In the light receiving portion 801, focusdetection pixels each provided with a plurality of PDs under one microlens are disposed, so as to each receive light beams that have passedthrough a divided exit pupil area of the taking lens 101. The readoutportion 802 performs analog digital signal processing using an A/Dconversion circuit, and adjustment (clamp processing) of a referencelevel.

The signal processor 803, on receipt of digital signal output from thereadout portion 103, outputs signals to the phase difference detectionportion 806 and the image processor 804 (discussed later). At this time,the signal processor 803, on receipt of the subject detection result ofthe subject detection portion 805, selectively outputs the focusdetection signals of the subject detection area.

The image processor 804 performs image processing such as defectivepixel correction, noise reduction, color conversion, white balancecorrection and gamma correction, resolution conversion processing, imagecompression processing, and the like on image generation signalsreceived from the signal processor 803. The subject detection portion805, on receipt of digital signal output from the image processor 804,determines the signal area for performing focus detection processing.

The phase difference detection portion 806 calculates a phase differenceevaluation value for performing focus detection employing a phasedifference detection method on focus detection signals received from thesignal processor 803. The output portion 807 outputs phase differenceevaluation values and digital signals for image generation that arereceived from the phase difference detection portion 806 and the imageprocessor 804 to outside of the image sensor 800. Similar effects to thethird embodiment are thus also obtained by the image sensor of theseventh embodiment of the present invention.

Incidentally, in image capture plane phase difference AF, one phasedifference evaluation value is calculated by performing a correlationoperation on the focus detection signals obtained from the plurality ofPDs of the plurality of pixels. In the third embodiment, the outputportion 106 output a plurality of digital signals for focus detectionthat are required in calculating this phase difference evaluation valueto outside of the image sensor 100. On the other hand, in the case ofthe seventh embodiment, the output portion 807 outputs the phasedifference evaluation value calculated by the phase difference detectionportion 806 to outside of the image sensor 800. In other words, theamount of signals that are output to outside of the image sensor 800 iseven less in the seventh embodiment than in the third embodiment.Accordingly, the output portion 807 of the image sensor 800 is able totransmit signals required in focus detection control in even less timethan the third embodiment, and is able to perform focus detectioncontrol by image capture plane phase difference AF at high speed.

Also, with the image sensor 800 of the seventh embodiment, the subjectdetection portion 805 performs subject detection, using digital signalsprocessed by the image processor 804. Thus, it is also possible toperform subject detection processing that also uses color information inaddition to using signal subjected to defective pixel correction andnoise reduction. Accordingly, the subject detection of the seventhembodiment can be performed with greater accuracy than the thirdembodiment.

Note that, with the seventh embodiment in the image sensor 800, evenmore constituent elements than the image sensor 100 of the thirdembodiment internally constitute the image sensor 800. Accordingly, theimage sensor 800 is preferably a stacked image sensor, such as is shownin FIG. 16 .

Eighth Embodiment

In the first to third embodiments, high-speed image capture plane phasedifference AF was described as being possible as a result of the imagesensor selectively outputting/transmitting focus detection signals.However, the applicable scope of the present invention is not limited toimage capture plane phase difference AF, and the present invention canalso be applied to automatic exposure control (image capture plane AE)that uses the signals of the image sensor. With image capture plane AE,specifically, control is performed after the image capturing apparatushas automatically determined the aperture of the taking lens, theaccumulation time of the image sensor, the sensitivity (gain) of theimage sensor, and the like. In the eighth embodiment of the presentinvention, the object is to perform faster image capture plane AE, as aresult of the image sensor selectively outputting/transmitting signalsto be used in image capture plane AE.

FIG. 19 is a block diagram showing the configuration of an imagecapturing apparatus of the eighth embodiment of the present invention.In FIG. 19 , the same reference signs are given to constituent portionsthat are same as the third embodiment, and a detailed descriptionthereof is omitted. The present embodiment differs from the thirdembodiment in that the image capturing apparatus is provided with alight metering portion 820.

The signal processor 105, on receipt of digital signal output for imagegeneration from the readout portion 103, outputs the signals to theoutput portion 106. The signal processor 105 selects digital signals forimage generation to be used in image capture plane AE, according to alight metering method set in the image capturing apparatus. At thistime, in the case of the so-called “spot metering method” and “partialmetering method”, for example, the signal processor 105 selects thesignals of a partial area in the middle of the screen, for example.Also, in the case of the so-called “evaluation metering method”, thesignal processor 105 selects the signals of the entire screen. Selectingsignals by row thinning or column thinning in a range in which theamount of signals required in calculation of a light metering evaluationvalue is obtained, however, rather than selecting the signals of all ofthe pixels of the entire screen, is preferable in order to performfaster image capture plane AE. Also, similarly to the third embodiment,a configuration may be adopted in which image generation signals of afocus detection area arbitrarily selected by the user and a subject areadetected by the subject detection portion 105 a are selected.

The light metering portion 820, on receipt of a digital signal for imagegeneration from the output portion 106, calculates a light meteringevaluation value for performing image capture plane AE. The digitalsignals for image generation that are input to the light meteringportion 820 are the signals of an area selected by the signal processor105 provided inside the image sensor 100. Accordingly, because the imagegeneration signals for image capture plane AE that the output portion106 of the image sensor 100 transmits to the light metering portion 820are only signals required for exposure control, the communication bandis used efficiently. Computational operation processing for lightmetering evaluation value calculation in areas that will not ultimatelybe required in image capture plane AE and processing for extractingsignal that will ultimately be required are also not required in theinternal processing of the light metering portion 820. Accordingly, thelight metering portion 820 is able to increase the processing speed atwhich the light metering evaluation value is calculated.

The lens controller 114, on receipt of the output of the light meteringevaluation value from the light metering portion 820, drives thediaphragm of the taking lens 101, based on the light metering evaluationvalue. Furthermore, the overall control/operation unit 109 drives theimage sensor 100 based on the light metering evaluation value, andcontrols the accumulation time and sensitivity (gain) of the imagesensor 100.

According to the eighth embodiment of the present invention, asdescribed above, because the image generation signals that the outputportion 106 of the image sensor 100 transmits to the light meteringportion 820 are only signals required for image capture plane AE,high-speed transmission is possible. The light metering portion 820 isnot required to perform computational operation processing for lightmetering evaluation value calculation that will not ultimately berequired in image capture plane AE or processing for extracting signalsthat will ultimately be required. Accordingly, faster image captureplane AE can be performed.

Note that, in the pixels of the image sensor 100 of the eighthembodiment, it is not necessarily the case that a plurality of PDs aredisposed per pixel as described in FIG. 2 . As long as there are signalsfor image generation (i.e., signals obtained by combining the signals ofa plurality of PDs under one micro lens on a pixel-by-pixel basis) inorder to calculate the light metering evaluation value, a configurationin which there is one PD under every one micro lens may be adopted.

Although, in the eighth embodiment, a configuration that applies imagecapture plane AE on the basis of the configuration of the thirdembodiment was described, image capture plane AE can also be similarlyapplied on the basis of the configuration of the seventh embodiment. Inthis case, the image sensor 800 is configured by replacing the phasedifference detection portion 806 in FIG. 18 with a light meteringportion.

Although preferred embodiments have been described above, the presentinvention is not limited to these embodiments, and various modificationsand changes that are within the spirit of the invention can be made.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application Nos.2015-183246 and 2015-183247, both filed Sep. 16, 2015 and 2016-144756,filed Jul. 22, 2016 which are hereby incorporated by reference herein intheir entirety.

What is claimed is:
 1. An image sensor comprising: a pixel portion inwhich a plurality of unit pixels each having one micro lens and aplurality of photoelectric conversion portions are arrayed in a matrix;a signal holding portion configured to hold signals corresponding toelectric charges accumulated in a part of the photoelectric conversionportions among the plurality of photoelectric conversion portions ineach of the unit pixels and signals corresponding to electric chargesobtained by combining electric charges accumulated in all of thephotoelectric conversion portions in each of the unit pixels; an outputportion configured to output signals based on the signals correspondingto electric charges accumulated in the part of the photoelectricconversion portions among the plurality of photoelectric conversionportions in each of the unit pixels and signals based on signalscorresponding to electric charges obtained by combining electric chargesaccumulated in all of the photoelectric conversion portions in each ofthe unit pixels, the signals being held in the holding portion, whereinthe pixel portion is formed on a first semiconductor substrate and thesignal holding portion is formed on a second semiconductor substratedifferent from the first semiconductor substrate, and wherein the firstsemiconductor substrate and the second semiconductor substrate arestacked with each other.
 2. The image sensor according to claim 1,wherein the output portion outputs signals based on the signalscorresponding to electric charges accumulated in the part of thephotoelectric conversion portions among the plurality of photoelectricconversion portions in each of the unit pixels arranged in a part of thepixel area held in the signal holding portion.
 3. The image sensoraccording to claim 2, wherein the output portion outputs signals basedon the signals corresponding to electric charges obtained by combiningelectric charges accumulated in all of the photoelectric conversionportions among the plurality of photoelectric conversion portions ineach of the unit pixels arranged in overall pixel area held in thesignal holding portion.
 4. The image sensor according to claim 2,wherein the part of the pixel area is a subject area.
 5. The imagesensor according to claim 2, wherein the part of the pixel area is anarea selected by a user.
 6. The image sensor according to claim 1,wherein the signal holding portion has a plurality of holding circuitsthat are each configured to hold signals readout from the pixel portionfor each unit pixel.
 7. The image sensor according to claim 1, whereinthe signal holding portion has a plurality of holding circuits that areeach configured to hold signals readout from the pixel portion for eachcolumn.
 8. An image capturing apparatus comprising: an image sensor; animage processor configured to process signals output from the imagesensor and generate an image; and a focus detector configured to processread signals and perform focus detection, wherein the image sensorincludes: a pixel portion in which a plurality of unit pixels eachhaving one micro lens and a plurality of photoelectric conversionportions are arrayed in a matrix; a signal holding portion configured tohold signals corresponding to electric charges accumulated in a part ofthe photoelectric conversion portions among the plurality ofphotoelectric conversion portions in each of the unit pixels and signalscorresponding to electric charges obtained by combining electric chargesaccumulated in all of the photoelectric conversion portions in each ofthe unit pixels; an output portion configured to output signals based onthe signals corresponding to electric charges accumulated in the part ofthe photoelectric conversion portions among the plurality ofphotoelectric conversion portions in each of the unit pixels and signalsbased on signals corresponding to electric charges obtained by combiningelectric charges accumulated in all of the photoelectric conversionportions in each of the unit pixels, the signals being held in theholding portion, wherein the pixel portion is formed on a firstsemiconductor substrate and the signal holding portion is formed on asecond semiconductor substrate different from the first semiconductorsubstrate, and wherein the first semiconductor substrate and the secondsemiconductor substrate are stacked with each other.
 9. The imagecapturing apparatus according to claim 8, wherein the output portionoutputs signals based on the signals corresponding to electric chargesaccumulated in the part of the photoelectric conversion portions amongthe plurality of photoelectric conversion portions in each of the unitpixels arranged in a part of the pixel area held in the signal holdingportion.
 10. The image capturing apparatus according to claim 9, whereinthe output portion outputs signals based on the signals corresponding toelectric charges obtained by combining electric charges accumulated inall of the photoelectric conversion portions among the plurality ofphotoelectric conversion portions in each of the unit pixels arranged inoverall pixel area held in the signal holding portion.
 11. The imagecapturing apparatus according to claim 9, wherein the part of the pixelarea is a subject area.
 12. The image capturing apparatus according toclaim 9, wherein the part of the pixel area is an area selected by auser.
 13. The image capturing apparatus according to claim 8, whereinthe signal holding portion has a plurality of holding circuits that areeach configured to hold signals readout from the pixel portion for eachunit pixel.
 14. The image capturing apparatus according to claim 8,wherein the signal holding portion has a plurality of holding circuitsthat are each configured to hold signals readout from the pixel portionfor each column.